The present invention relates to electronic devices, and more particularly to image reformatting methods and related devices such as digital cameras.
Recently, digital cameras have become a very popular consumer appliance appealing to a wide variety of users ranging from photo hobbyists, web developers, real estate agents, insurance adjusters, photo-journalists to everyday photography enthusiasts. Advances in large resolution CCD/CMOS sensors coupled with the availability of low-power digital signal processors (DSPs) has led to the development of digital cameras with both high resolution image and short video clip capabilities, and these capabilities have spread into various consumer products such as cellular phones. The high resolution (e.g., sensor with a 2560×1920 pixel array) provides quality offered by traditional film cameras. U.S. Pat. No. 5,528,293 and U.S. Pat. No. 5,412,425 disclose aspects of digital camera systems including storage of images on memory cards and power conservation for battery-powered cameras.
FIG. 3a is a functional block diagram for digital camera control and image processing; the automatic focus, automatic exposure, and automatic white balancing are referred to as the 3A functions. The image processing typically includes functions such as color filter array (CFA) interpolation, gamma correction, white balancing, color space conversion, and JPEG/MPEG compression/decompression (JPEG for single images and MPEG for video clips) and is referred to as the image pipeline. Note that the typical color CCD consists of a rectangular array of photosites (pixels) with each photosite covered by a filter (CFA): red, green, or blue. In the commonly-used Bayer pattern CFA one-half of the photosites are green, one-quarter are red, and one-quarter are blue. FIG. 3b illustrates possible hardware components for a digital camera.
The current trend of incorporating video capabilities into high resolution digital cameras creates a problem because the camera must satisfy both the high resolution of a still image camera and the high frame rate/low resolution requirements of a video camera. Consequently, most image sensors (CCD or CMOS) employ schemes to average pixel values within the sensor device for video mode. Averaging pixels does two things:                It reduces the number of pixels per frame so that a desired video frame rate can be achieved without straining the pixel rate. For example, a 5 megapixel sensor (2560×1920 pixel array) which can internally reduce output to VGA (640×480) resolution will achieve 30 frames per second needed for video with a 9 Mpixels/second output rate. Without such internal averaging the device will need an output rate of 150 Mpixels/second rate to achieve 30 frames/second, and this is much faster than current analog front ends (AFEs), which include analog-to-digital conversion, can operate.        It reduces the noise. Noise introduced in the sensor and the signal path can be reduced through averaging.        
The averaging of pixel values in the sensor device, however, poses challenges for the subsequent image pipeline processing of the pixel data. Indeed, the sensor often outputs video-mode data in some fixed, regular, but locally scrambled format and not in normal raster-scan order. The video format is not consistent across various sensor manufacturers. Thus it is a problem of image pipelines to support all the different video output formats, including adaptation to future formats.